Back

JAMIA Open

◐ Oxford University Press (OUP)

Preprints posted in the last 30 days, ranked by how well they match JAMIA Open's content profile, based on 42 papers previously published here. The average preprint has a 0.07% match score for this journal, so anything above that is already an above-average fit.

1
An LLM-Based Auditing Framework for Targeted Quality Assurance in Cancer Registries

Seifaddini, M.; Beheshti, M.; Steffens, J.; Carnagey, D.; Esebua, M.; Wakefield, M.; Zachary, I.

2026-09-23 health informatics 10.64898/2026.09.22.26363595 medRxiv
Top 0.1%
15.1%
Show abstract

Cancer registries are essential for population-level surveillance, epidemiologic research, and public health planning, but their accuracy depends on human review of key variables documented in unstructured clinical text. This is time-consuming, and current quality assurance (QA) typically relies on reviewing fewer than 10% of records at random, which can miss facility- or time-specific errors. We developed a large language model (LLM)-based framework for targeted cancer registry QA. The framework uses LLMs to extract biomarker values from clinical text, compares them with registry entries, and flags disagreements for expert review. Six open-source LLMs were benchmarked locally in a privacy-preserving environment, and the best model per site and report type was applied to breast and prostate cancer, two of the most common cancers in the United States. For the four breast cancer biomarkers ER, PR, HER2, and Ki-67, the disagreement rate against quality-assured registry values in 1,852 breast cancer cases was lowest for ER at 1.67% and highest for Ki-67 at 9.18%. In 920 prostate cancer cases, the disagreement rate with the deployed models stayed below 8% for every categorical Gleason variable, while the exact-value PSA metric reached 17.28%. PSA therefore generated the largest volume of candidate discrepancies, though individual PSA flags require more corroboration than categorical flags. Applied to a full year of records (11,740 breast and 6,792 prostate cases), the framework flagged disagreements and assigned each a probable cause, grounded in NAACCR coding rules, to prioritize expert review. Extracted values are linked to their supporting text, and model, prompt, and pipeline versions are logged for reproducibility. By directing review toward records most likely to contain discrepancies, the framework offers an efficient, auditable alternative to random QA sampling for cancer registries.

2
Improving MASLD Identification in Patients with T2D: A Computable Phenotype Framework Integrating Structured Data and Clinical Notes

Chen, Y.; Sharma, A.; Yang, Q.; Chen, Y.; Yin, R.

2026-10-02 health informatics 10.64898/2026.09.26.26364078 medRxiv
Top 0.1%
14.8%
Show abstract

Metabolic dysfunction-associated steatotic liver disease (MASLD) is highly prevalent among individuals with type 2 diabetes (T2D), yet accurate identification from electronic health records (EHRs) remains challenging because clinically relevant information is fragmented across structured data and unstructured clinical documentation. We developed a rule-based computable phenotype (CP) framework that integrates structured EHRs with clinical notes for scalable MASLD identification in patients with T2D. The framework comprises five complementary CP algorithms derived from validated fibrosis indices and liver injury biomarkers, together with a lightweight natural language processing module that identifies MASLD-related evidence using keyword matching, contextual filtering, and negation detection. The proposed framework was systematically evaluated across multiple threshold settings, with additional validation through age- and sex-stratified analyses and manual chart review. The findings demonstrate that integrating structured and unstructured EHR data provides an accurate and scalable framework for real-world MASLD phenotyping and offers a practical strategy for developing CPs for other chronic diseases.

3
A Modular Architecture for Dynamic Evaluation of Best-Practice Care

Hoang, M. T.; Shaw, T.; Igasto, C.; Donnelly, C.; Shetty, A.; Pradhan, M.

2026-10-06 health informatics 10.64898/2026.10.03.26364324 medRxiv
Top 0.1%
10.5%
Show abstract

Objective: Retrospective evaluation of clinical pathway adherence using electronic medical record (eMR) data is limited by heterogeneous data structures and statistical measures that do not account for patient-specific context, care sequence, or timing. We designed a modular event-based architecture to dynamically evaluate the New South Wales Adult Sepsis Pathway over time. Method: Data from eMRs were collected from adult emergency department encounters across four hospitals between July 2022 and June 2024 and processed through three sequential layers: event-stream transformation, pathway-relevance filtering, and state-based pathway evaluation. Heterogeneous records were standardized and organized into a chronological timeline for each encounter. The framework then updated the pathway state as new clinical information became available and assessed whether recommended care was delivered within the relevant timeframe and sequence. Result: The architecture processed 451,124 encounters. This included 57,631,084 clinical events extracted and transformed into encounter-level event streams. After filtering for pathway relevance, 43,951,146 events remained (76.3%). A total of 129,907 (34.6%) encounters entered the sepsis pathway, 50,118 (38.6%) entered YELLOW state, indicating progression to suspected sepsis, and 3,349 (2.6%) encounters progressed to RED state from GREEN, indicating the initial shock. Of 50,118 encounters entering YELLOW state, 2,426 (4.8%) progressed to shock, 8,700 (17.4%) encounters were recognized with sepsis through a blood culture order, and 4,287 (8.6%) encounters were recognized with sepsis through an antibiotic order. This demonstrates the ability of the architecture to prioritize higher-acuity states during evaluation rather than going through an intermediate state. Conclusion: The architecture developed shows the feasibility of transforming fragmented routine eMR data into a reusable longitudinal representation that supports context-dependent pathway evaluation. Separating data preparation from pathway rules improves transparency, maintainability, and adaptability and provides a foundation for future interoperable, near-real-time pathway monitoring

4
Design and Technical Evaluation of a Standards-Based System That Delivers Enclave-Derived Case-Finding Models to Patients and Clinicians

Asiaee, A.; Aryan, K.

2026-10-21 health informatics 10.64898/2026.10.20.26365982 medRxiv
Top 0.1%
9.7%
Show abstract

Background. Research data enclaves such as the All of Us Research Program do not allow participant-level data, or models fitted to them, to leave, which makes it hard to deliver case-finding models, which flag patients with possibly uncoded conditions for clinician review, to patients and clinicians. Objectives. We describe and technically evaluate a system that fits models outside an enclave from released summary counts and delivers them through standard health information technology interfaces. Methods. We exported dissemination-compliant summary counts from the All of Us Researcher Workbench and, outside the enclave, fitted versioned models for Long COVID, chronic fatigue, fibromyalgia, endometriosis, Lyme disease, lupus, migraine, depression, and anxiety. A model contract (a model file and a written specification), a Python reference implementation, and synthetic Fast Healthcare Interoperability Resources (FHIR) test patients define the expected outputs. One JavaScript scorer runs in a patient demonstration, a SMART on FHIR clinician application, and a CDS Hooks service. An atlas displays aggregate results. We evaluated the system on synthetic records with automated tests, cross-implementation output comparison, launches through the public SMART sandbox, and a counterexample review of the scoring and display rules. Results. The deployed Long COVID model had an area under the receiver operating characteristic curve of 0.67 in a random validation split. The eight self-report models reached 0.61 to 0.82 in identifying self-reporters without a code. The JavaScript scorer reproduced all 162 reference outputs exactly, and all 437 automated tests passed. A counterexample review found 5 classes of failure, including scoring ineligible patients and treating unavailable data as absent, all fixed and added as tests. Conclusions. Models fitted from released summary counts can be delivered through FHIR, SMART on FHIR, and CDS Hooks with reproducible scoring on synthetic records. Real-record interoperability, usability, and prospective clinical performance remain to be evaluated.

5
Considerations for implementing a machine learning model for suicide risk in the health systems that serve American Indian communities

Dayal, R. R.; Goklish, N. A.; Adams, R.; Rebman, P.; Suttle, R.; Garcia, M.; Mega, J.; de la Sierra, V. Q.; Haroz, E. E.

2026-10-07 health informatics 10.64898/2026.10.05.26364799 medRxiv
Top 0.2%
8.0%
Show abstract

Objective: We explored the implementation considerations of machine learning (ML) suicide risk identification in a health system that serves primarily American Indian patients in the Southwestern United States. Previous work developed and reported the performance of suicide risk models, but traditional performance metrics do not directly reflect clinical workflows. Materials and Methods: Using EHR data collected for Emergency Department (ED) visits between 10/01/2019 and 10/02/2021, we conducted Decision Curve Analysis (DCA) to identify the standardized net benefit (SNB) of all models previously evaluated and plotted the potential workload implications of the model with the greatest SNB. Finally, we evaluated suicide screening practices during the study period to inform implementation barriers. Results: Among 9,244 patients, 139 (1.5%) had a suicide attempt or death within 90 days of an included visit. The population-specific logistic regression model had the greatest SNB compared to other approaches. When operationalized, the 95th percentile cutoff did not substantially increase clinical workloads, while the 75th percentile cutoff substantially impacted workloads. The secondary analysis on suicide screening depicted variations in screening administration over time. Discussion: While DCA identified advantages of ML suicide risk identification over current standards, the complexities of suicide-related care limited its clinical interpretability. A stepped-care framework may mitigate potential workload burdens. Training initiatives and dynamic implementation strategies will support high-quality implementation of ML suicide risk identification. Conclusion: Our analyses support the feasibility and utility of augmenting existing suicide screening practices with ML suicide risk identification, and demonstrate the importance of nuanced analyses to inform implementation considerations.

6
AI-Empowered Multimodal Data Integration and Knowledge Discovery for Pharmacovigilance

Li, Y.; Zhou, H.; Yeh, F.-Y.; Sun, Z.; Zheng, J.; Alzaidi, S.; Guan, H.; Li, J.; Seger, D. L.; Wang, L.; Li, M.; Li, X.; Chen, T.; Plasek, J. M.; Li, H.; Jiang, A.; Hong, P.; Li, F.; Zhang, W.; Cui, L.; Bates, D. W.; Zhou, L.; He, Y.; Tao, C.

2026-10-21 health informatics 10.64898/2026.10.20.26365969 medRxiv
Top 0.2%
7.8%
Show abstract

Biomedical artificial intelligence (AI) increasingly relies on heterogeneous data sources, yet differences in data structure, terminology, evidence quality, and provenance complicate their integration into reliable knowledge resources. Here, we develop an evidence-aware framework for processing and integrating heterogeneous pharmacovigilance data from FDA Adverse Event Reporting System (FAERS) reports, biomedical literature, clinical trials, and electronic health records in Mass General Brigham. The framework combines large language model (LLM)-based information extraction, ontology-driven semantic normalization, cross-source evidence assessment, statistical and machine learning-based prioritization, and ingredient-based adverse drug event (ADE) relationship inference to transform heterogeneous evidence into a structured and evidence-based knowledge resource. The constructed knowledge system contains 11,588 drug nodes, 44,843 ingredient nodes, 3,275 ADE nodes, 993 indication nodes, and more than 2.47 million relationships, with each drug-ADE relationship assigned a confidence score. We further developed a drug-level web interface for accessing the knowledge resource and validated its utility in downstream AI tasks. Specifically, incorporating the developed knowledge resource improved the comprehensiveness, relevance, and overall quality of LLM-based drug-specific ADE synthesis and improved the precision and F1 score of clinical ADE relation extraction. These results demonstrate that AI-assisted processing combined with evidence-aware curation can transform heterogeneous biomedical evidence into reusable, AI-ready knowledge with explicit provenance and evidence strength information. The framework provides a general strategy for integrating heterogeneous biomedical data to support reproducible knowledge construction and downstream biomedical AI applications.

7
A Framework to Monitor Editing of Artificial Intelligence-Generated Medical Documentation

Garcia-Agundez, A.; Zhou, S.; Pourian, J.; Blebea, C.; Daneshpajouhnejad, P.; Dente, E.; Shi, K.; Pollet, S.; Xia, F.; Shi, X.; Thombley, R.; Fenton, C.; Murray, S. G.; Adler-Milstein, J.; Schmajuk, G.; Feng, J.; Yazdany, J.

2026-10-05 health informatics 10.64898/2026.09.30.26364427 medRxiv
Top 0.2%
7.7%
Show abstract

Objective: To develop and evaluate a framework for characterizing clinician editing of Artificial Intelligence (AI)-generated[KS1.1] documentation and assess its feasibility for health system-level monitoring of AI scribes. Materials and Methods: We analyzed outpatient encounters in which an AI scribe was used at a single academic health system, examining the History of Present Illness (HPI) and Assessment and Plan (A&P) sections of notes. We characterized edits on three dimensions[KS2.1][AG2.2]: lexical edit intensity based on Levenshtein distance, embedding edit intensity BERT[KS3.1]Score, and clinical edit intensity based on removed and added UMLS[KS4.1][AG4.2] concepts.[KS5.1][AG5.2] We analyzed the Positive Predictive Value (PPV) of clinical edit intensity as a measure of clinically meaningful editing using clinicians as the gold standard, examined correlations among dimensions, and designed exponentially weighted moving-average control charts to monitor longitudinal changes in clinician editing behavior.[KS6.1][AG6.2] Results: 268,379 encounters were included (267,654 HPI, 267,594 A&P). Clinical edit intensity [≥]1 had an 88.9% PPV for clinically meaningful editing. Lexical and embedding edit intensity were highly correlated (Spearman {rho} 0.95), while clinical edit intensity was less strongly correlated with both ({rho} 0.77-0.81). Longitudinal monitoring detected changes coinciding with system-wide rollout.[KS7.1][AG7.2] Discussion:[KS8.1][AG8.2] Clinicians edited A&Ps more heavily than HPIs, potentially reflecting greater attention to content involving clinical decision-making. Over one-third of sections involved clinical concept changes, and clinical edit intensity identified clinically meaningful edits while providing information complementary to lexical editing measures. Conclusion: Clinician editing can be characterized at scale using complementary editing dimensions, providing a scalable signal for post-deployment surveillance of the human-AI documentation process.

8
Leveraging Large Language Models for Temporal PHI De-identification in Real-World Clinical Notes

Wang, X.; Wang, L.; Wen, A.; Li, R.; Lu, S.; Hu, Y.; Li, X.; Lyu, H.; Liu, H.

2026-09-23 health informatics 10.64898/2026.09.22.26360316 medRxiv
Top 0.2%
7.1%
Show abstract

De-identifying temporal protected health information (PHI) is challenging because temporal expressions appear in diverse formats and must be transformed while preserving clinically meaningful timelines. This study proposes and evaluates an LLM-based framework for temporal PHI de-identification in real-world clinical notes. Using 1,148 notes from 30 sarcoma patients (28,431 annotated temporal entities), we evaluated multiple modern LLMs for temporal entity extraction and surrogate generation. Proprietary models achieved strong extraction performance, with GPT-4o achieving the best overall results. Most models preserved temporal formatting (>99%), but surrogate generation remained challenging. GPT-5.4 achieved the highest shift correctness (90.2% on a shared-entity subset) and the lowest order violations. Error analysis revealed systematic shift deviations ({+/-}1, {+/-}30/31, {+/-}365 days), highlighting persistent limitations in LLM temporal reasoning. These findings suggest that reliable temporal de-identification will require hybrid or multi-agent approaches beyond standalone LLMs.

9
BanglaDiaHallu: Hallucinations of large language models and the effect of guideline-grounded retrieval-augmented generation in Bangla diabetes question answering

Nahid, K.; Anam, M.; Saon, M. T. H.; Any, O. H.; Hassan, K. M. R.; Anwar, M. M.

2026-10-16 health informatics 10.64898/2026.10.14.26365587 medRxiv
Top 0.2%
6.9%
Show abstract

Large language models (LLMs) are increasingly consulted for health advice, but they can produce fluent statements that are false or unsupported, known as hallucinations, and this risk is poorly characterised in low-resource languages such as Bangla. We measured hallucinations of locally deployable open-weight LLMs in Bangla diabetes question answering and tested whether retrieval-augmented generation (RAG) grounded in Bangladeshs National Guideline on Diabetes Mellitus reduces them. Forty patient questions spanning six clinical sub-domains were each answered twice, under zero-shot prompting, by Qwen3.5-9B, TigerLLM-9B, Qwen3.6-35B-A3B and Gemma-4-26B-A4B, and by BanglaDiaRAG, which adds hybrid dense (BGE-M3) and sparse (BM25) retrieval from a physician-reviewed Bangla guideline knowledge base to Qwen3.5-9B. Two physicians rated the 400 de-identified responses by consensus for medical accuracy, completeness, safety, readability and reliability on a 0-1 scale, and responses with accuracy below 0.70 were classified as hallucinated. Hallucination rates were 2.5% for Gemma-4-26B-A4B, 3.8% for TigerLLM-9B, 22.5% for Qwen3.6-35B-A3B and 77.5% for Qwen3.5-9B. Guideline grounding lowered the rate of Qwen3.5-9B to 11.3% (difference -66.3 percentage points; 95% CI -76.3 to -55.0) and raised every criterion by 0.24 to 0.38 points (Holm-adjusted p < 0.001). BanglaDiaRAG outscored Qwen3.6-35B-A3B on all five criteria and matched or exceeded TigerLLM-9B, but it remained below Gemma-4-26B-A4B in safety, readability and reliability, and its residual errors were specific, confident and potentially harmful. Guideline grounding can turn a weak general-purpose model into a much safer Bangla diabetes assistant, but the choice of base model mattered as much as retrieval and no configuration was free of hallucination, so clinical oversight remains essential before patient-facing use.

10
ClinStigma: A Clinician-Guided Framework for Developing Synthetic Benchmarks of Contextual Stigmatizing Language in Clinical Documentation

Izzo, J. A.; McIntyre, A. M.; Nguyen, J.; Bashaw, D.; Torrance, C. A.; Foster, J.

2026-10-12 health informatics 10.64898/2026.10.10.26365233 medRxiv
Top 0.2%
6.8%
Show abstract

Objective Large language models (LLMs) are increasingly used to generate, summarize, and evaluate clinical documentation, yet standardized methods for evaluating nuanced, context-dependent clinical language remain limited. Existing approaches rely primarily on annotated clinical corpora or keyword-based methods, restricting reproducibility, public dissemination, and systematic comparison across models. We present ClinStigma, a clinician-guided framework for constructing, annotating, and evaluating synthetic benchmarks of contextual stigmatizing language, demonstrated through ClinStigma-SUD, its inaugural implementation for substance use disorder documentation. Materials and Methods Candidate clinical sentences were generated using a frontier language model and iteratively reviewed according to a clinician-developed annotation rubric refined through expert review in stigmatizing language. The resulting benchmark intentionally incorporates explicit stigmatizing, neutral, and contextually ambiguous ("gray-zone") documentation requiring clinical reasoning rather than keyword recognition. Three locally deployed open-weight LLMs were evaluated using zero-shot inference, followed by external evaluation using independently annotated clinical documentation from MIMIC-IV. Results ClinStigma-SUD consists of 500 clinician-adjudicated synthetic clinical documentation sentences. Inter-rater agreement was 98.8% across the full benchmark (Cohen's {kappa} = 0.976) and 94.0% within the intentionally ambiguous subset ({kappa} = 0.880). Under zero-shot prompting, binary accuracy ranged from 84.8% to 91.8% across evaluated models, while F1 for the ambiguous (gray) class ranged from 0.038 to 0.423. Zero-shot external evaluation on 3,716 annotated examples from the independently annotated Harrigian MIMIC-IV corpus yielded 84.4% accuracy (84.4% balanced accuracy; macro F1 0.824), against a 69.9% majority-class baseline. Stigmatizing-class F1 was 0.879 in the substance-related subgroup and 0.883 in the full external corpus. Discussion and Conclusion ClinStigma establishes a reproducible clinician-guided framework for developing synthetic evaluation benchmarks for contextual clinical language. By combining structured clinical expertise, explicit annotation guidance, synthetic benchmark generation, and external evaluation, the framework enables open, privacy-preserving evaluation of language models while providing a scalable foundation for benchmark development across additional clinical specialties, including obstetrics, emergency medicine, and mental health.

11
Self-reported diagnoses as training labels for finding uncoded chronic fatigue, fibromyalgia, and endometriosis in the All of Us Research Program

Asiaee, A.; Aryan, K.

2026-10-20 health informatics 10.64898/2026.10.19.26365869 medRxiv
Top 0.2%
6.8%
Show abstract

Objective: Case-finding models for underdiagnosed conditions are usually trained on diagnosis codes, which are missing from the records of many patients with a clinical diagnosis. We asked whether models trained on self-reported diagnoses rank uncoded self-reporters higher than models trained on later codes, and how a score fit only from released summary counts compares with a participant-level model. Methods: In the All of Us Research Program, we followed survey respondents with no qualifying diagnosis code by the survey date for later codes. For chronic fatigue, fibromyalgia, and endometriosis (participants recorded as female at birth), we fit a naive Bayes score with empirical-Bayes shrinkage, from released feature-by-label counts for the self-reported diagnosis and from exact counts inside the data enclave for a later code, and an elastic net on participant-level data for each label. In a random 20% validation split, we measured discrimination of self-report, of uncoded self-reporters (no qualifying code observed), and of later codes. Results: For chronic fatigue (149,860 respondents, including 7,770 self-reporters, 95% with no code observed), the summary-count score trained on self-report ranked uncoded self-reporters with an area under the receiver operating characteristic curve (AUROC) of 0.74 (95% CI, 0.72-0.75), against 0.67 (95% CI, 0.65-0.69) when trained on later codes. For later codes, the two scores were similar (0.69 vs 0.70). Fibromyalgia showed the same pattern (0.73 vs 0.64 and 0.74 vs 0.72). Elastic nets trained on self-report ranked uncoded self-reporters higher than the summary-count scores (0.80 vs 0.74 and 0.78 vs 0.73) but discriminated later codes less well than code-trained elastic nets. For endometriosis, the self-report score also ranked uncoded self-reporters above the code-trained score (0.61 vs 0.53), but discrimination was low and it did not discriminate later codes (0.50 vs 0.70). Conclusion: The choice of training label changes which patients a case-finding model ranks highly. For all 3 conditions, self-reported diagnoses were the more useful label for finding uncoded self-reporters. For chronic fatigue and fibromyalgia, the self-report score also discriminated later codes as well as the code-trained score. Independent clinical validation is needed before these models guide case finding.

12
Parametric Models for Predicting Gram-negative Antibiotic Susceptibility Phenotype: A Brief Report

Garg, A.; Graham, C.; Daneman, N.; MacFadden, D.

2026-10-02 infectious diseases 10.64898/2026.10.01.26363073 medRxiv
Top 0.2%
6.6%
Show abstract

Predicted probabilities of antibiotic susceptibility to known or suspected pathogens can be used to support antibiotic selection. Patient demographic, clinical, and prior microbiologic history can be used to generate location-specific prediction models. For three Canadian tertiary care institutions we fit and evaluated models for predicting Gram-negative antibiotic susceptibilities to common antimicrobial agents based on previously established model structures. A web-based application was developed to permit visualization of antibiotic susceptibility predictions and support antibiotic decision-making.

13
Introducing Infrastructure as Code for deployment and governance of medical AI: A case study

Vogel, M.; Reuter, N.; Ruf, S.; Noori, A.; Madrid, J.; Balicer, R.; Witzenrath, M.; Sander, L. E.; Groeschel, M. I.

2026-10-13 health informatics 10.64898/2026.10.12.26365327 medRxiv
Top 0.2%
6.6%
Show abstract

Clinical AI deployment rarely reaches the bedside because both workstation integration and governed backend provisioning are challenging. We introduce a workflow that allows rapid and governed access to AI functionalities through a browser interface or API, enabling privacy-compliant data analysis. The complex setup for compliant healthcare applications such as a secure backend, private AI model endpoint, and managed identity configuration is encoded by itself, a concept known as Infrastructure as Code (IaC). IaC is a concept in which networks, access rights, and service endpoints are written down as machine-readable files that can be reviewed once and executed identically in any institution. Here, we apply this concept to a clinical use case. The process of deploying a governed, private clinical AI solution using this approach in a tertiary care center took, on average, 310.3 +- 196.6 seconds (n=25, mean +- SD). To further demonstrate the clinical value of this approach, we developed and deployed an LLM-based inconsistency checker, which identifies textual inconsistency in medical documents. When evaluating the tool on a set of 1,347 physician-approved clinical documents, the tool flagged a clinically severe inconsistency in 105 clinical documents (7.8%, approximately 1 in 13). In a severity-stratified physician audit, 149 of 150 sampled findings were confirmed (population-weighted PPV 98.2%; CI 94.8 - 100.0). In summary, we present the application of IaC frameworks to a healthcare use case and demonstrate that clinically useful AI applications can be deployed to physician workstations in hospital-governed cloud environments within a few minutes.

14
Uncoded Clinical Features from Multilingual Electronic Health Records in Catalonia: Development and Validation Study

Ouchi, D.; Riudavets, S.; Franquet, A.; Fernandez-Garcia, S.; Llor, C.; Moragas Moreno, A.; Giner-Soriano, M.; Torres, F.; Morros, R.

2026-09-22 health informatics 10.64898/2026.09.16.26363030 medRxiv
Top 0.3%
6.5%
Show abstract

Background: Unstructured free-text narratives in electronic health records (EHRs) contain critical clinical information that is not captured by structured standard codes. In Catalonia, primary care notes follow a semi-structured format known as MEAP (in Catalan). However, extracting structured variables using cloud-hosted commercial large language models (LLMs) raises substantial data privacy concerns and incurs high operational costs. Objective: To develop and validate a privacy-preserving local hybrid pipeline combining a lightweight small language model (SLM) with deterministic regular expressions for extracting six uncoded urinary tract infection (UTI) clinical features (fever, nitrites, leukocytes, lumbar pain, abdominal pain, and haematuria) from primary care EHR narratives. Methods: We deployed the open-weight model (Phi4-mini, 3.8B parameters) within the institutional firewall using Ollama. Prompts were iteratively refined in collaboration with clinicians, and regular expressions were applied post-inference. Validation was conducted across two independent arms: (1) a double-blind clinician gold standard review of 60 real patient records (consensus-resolved cases), and (2) an adversarial synthetic dataset of 720 notes enriched with linguistic noise, generated using GPT-4.1 and Grok-4.1. Point estimates and 95% confidence intervals (CIs) were computed for key performance metrics. Results: The pipeline processed 15,498 MEAP narratives from a matched cohort of 2,962 primary care patients and identified 4,663 clinical feature occurrences. Patients progressing to acute pyelonephritis (cases) presented a higher features burden than non-progressing controls (71.3% vs. 55.8%; standardized mean difference [SMD] = 0.326), particularly for fever (33.0% vs. 9.0%; SMD = 0.616) and lumbar pain (29.0% vs. 9.8%; SMD = 0.502). In real-world clinician validation, overall performance yielded 93.8% accuracy (95% CI 90.6%-96.1%), 83.6% sensitivity (95% CI 73.0%-91.2%), 96.6% specificity (95% CI 93.6%-98.4%), 87.1% positive predictive value (PPV; 95% CI 77.0%-93.9%), and 95.5% negative predictive value (NPV; 95% CI 92.3%-97.7%). In synthetic stress-testing, the pipeline demonstrated 87.2% accuracy (95% CI 84.6%-89.6%), 73.9% sensitivity (95% CI 68.9%-78.5%), near-perfect specificity (99.5%, 95% CI 98.1%-99.9%), and 99.2% PPV (95% CI 97.2%-99.9%). Conclusions: A privacy-preserving local hybrid framework combining a compact open-weight SLM with regular expression rules achieves high specificity and precision for extracting six uncoded clinical features from multilingual primary care narratives. Running entirely within institutional servers, this approach keeps patient data secure, meets privacy standards, and avoids Application Programming Interface (API) costs, making it a practical tool for EHR research, though with lower sensitivity for narratively complex descriptions.

15
Decision models can offload structured clinical tasks from large language models

Ariel, D.; Laban, G.; Endelman, S.; Balicer, R.; Dagan, N.; Cohen, S.

2026-10-12 health informatics 10.64898/2026.10.09.26365130 medRxiv
Top 0.3%
6.3%
Show abstract

Decision models like Jev read text and return probabilities, categories, or ordinal levels, with confidence scores for the latter two, at a fraction of large language models' (LLMs) cost. We explored how Jev could be integrated into clinical workflows for all these output types: mortality risk estimation, categorical Choosing-Wisely decisions for patient questions, and ordinal colon cancer staging. With standard LLM-style inputs, Jev trailed five flagship LLMs: mortality AUROC was 0.822 versus 0.837-0.846; Choosing-Wisely accuracy, 87.7% versus 98.4-100%; and staging accuracy, 45.4% versus 94.2-100%. Developer-recommended input alterations improved staging accuracy to 75.8% with medical context, 82.1% with task decomposition, and 85.8% with both. None of Jev's staging errors had full confidence. Using Jev for fully confident staging decisions and LLMs for the rest was non-inferior to each LLM alone and cut flagship LLM costs by 33-44%. Prompted as designed, decision models may efficiently share structured clinical tasks with LLMs.

16
Extraction of clinical information from faxed medical records using a small local large language model pipeline on consumer hardware

Tang, Z.

2026-10-05 health informatics 10.64898/2026.10.03.26364667 medRxiv
Top 0.3%
6.2%
Show abstract

Objective Specialty referral packets often arrive by fax and clinicians must manually read and organize a large amount of disparate information to affect continuity of care. Commercial large language models (LLMs) can organize packet information, but protected health information (PHI) must be secured. We evaluated whether an open-weight pipeline running entirely on one consumer graphics card can accurately extract clinical information from faxed referral packets. Materials and Methods We generated 200 synthetic colorectal cancer referral packets from simulated patient records, rendered them as faxes without a text layer. A local pipeline transcribed each packet with an optical character recognition model (OvisOCR2) and extracted structured data with an open-weight LLM (Qwen3.8 27B). Twenty outcomes spanning patient identifiers, laboratory results, dates, diagnoses, procedures, medications, imaging, and event chronology were scored against ground truth on 140 validation packets. Results Laboratory values were extracted exactly in 98.9% of 7879 instances, dates in 99.7%, diagnoses in 100%, and procedures in 92.9%. Timeline recall was 96.7% and precision 99.4%. Weighted accuracy was 0.990 (95% CI 0.987 - 0.992). None of 11,993 extracted facts were fabricated. Median processing time was 90 seconds per packet. Conclusion An open-weight pipeline on consumer hardware accurately extracted multi-domain clinical information from synthetic faxed referrals.

17
Learning the electronic health record at the minute-scale

Narang, K.; Addo, N.; Bains, J.; Firnberg, M.; Tat, S.; Hooft, A.; Arrubla, D. C.; Abed, B.; Singh, C.; Kamaleswaran, R.; Feng, J.; Kornblith, A. E.

2026-10-06 emergency medicine 10.64898/2026.10.04.26364562 medRxiv
Top 0.3%
6.2%
Show abstract

Electronic health record foundation models are traditionally trained on the scale of years, days, or hours. These timescales, however, lack the minute-scale resolution required to directly guide clinical decisions at the bedside. We introduced minute-scale learning, a new paradigm for training and evaluating models, and developed MINT, a minute-scale foundation model for pediatric emergencies. MINT was pretrained and validated on 766,733 pediatric emergency department visits at five health systems, comprising 16 years of data from 10 hospitals. On minute-scale forecasting tasks, MINT outperformed and generalized to external health systems better than task-specific models (superior in 21 of 25 comparisons). MINT outperformed physicians in forecasting escalations of respiratory support. MINT demonstrated uniquely minute-scale capabilities including department-scale monitoring, dynamic risk explanations, individualized physiologic response forecasts, and hypothesis generation. Learning at the minute-scale improves performance, strengthens generalizability, and provides insights, actionability, and scientific capabilities that are not accessible at other timescales.

18
Electronic Health Record Coding of Self-Reported Diagnoses in the All of Us Research Program

Asiaee, A.; Aryan, K.

2026-10-20 health informatics 10.64898/2026.10.18.26365863 medRxiv
Top 0.3%
6.2%
Show abstract

Importance: Conditions that are hard to diagnose, several more common in women, may be missing from electronic health records (EHRs), hiding patients from research cohorts and case finding. Objective: To measure how often the EHR contains a diagnosis code for chronic conditions that participants report, and to compare this with national survey benchmarks. Design, Setting, and Participants: This cross-sectional study used the All of Us Research Program Registered Tier Dataset v9. Participants were adults with EHR visits in both 12-month periods from July 2021 to June 2023 who completed a health history survey. We analyzed the data in October 2026. Exposures: Self-reported clinician diagnosis of 10 conditions with ongoing visits or treatment, as well as sex at birth, race and ethnicity, age, and years of EHR records. Main Outcomes and Measures: Share of participants reporting current care who had a qualifying diagnosis code, alone and relative to hypertension and type 2 diabetes, and the ratio of age-standardized coded shares to national survey prevalence. Results: Of 161,480 respondents, 114,610 (73,870 female [64.5%], median [IQR] age 63 [50-72] years) had at least 5 years of EHR records before the survey. Among those reporting current care, 84.6% (95% CI, 84.2%-85.0%) with hypertension and 87.0% (95% CI, 86.5%-87.6%) with type 2 diabetes had a code, compared with 41.3% (95% CI, 38.9%-43.8%) for endometriosis, 39.4% (95% CI, 34.3%-44.8%) for Lyme disease, and 24.4% (95% CI, 23.2%-25.6%) for chronic fatigue. Counting ICD-9-CM and other mapped codes, 55.3%, 51.5%, and 28.3%, respectively, had any qualifying record. Women had lower coded shares than men for hypertension, type 2 diabetes, obesity, depression, and anxiety (crude ratios 0.93-0.96, within-health-system ratios 0.98-1.02) and higher shares for migraine and fibromyalgia (within health systems, 1.11 [95% CI, 1.07-1.15] and 1.41 [95% CI, 1.29-1.57]). Age-standardized coded shares among active patients were 1.1 to 2.4 times national prevalence of the same self-reported diagnoses. Conclusions and Relevance: In this cross-sectional study, fewer than half of participants reporting ongoing care for chronic fatigue, Lyme disease, or endometriosis had a qualifying ICD-10-CM code. Linking each participant's report to their own record measured this gap directly.

19
Scalable Causal-Interpretable Machine Learning for Cancer Prescreening Using Electronic Health Records

Zhang, S.; Xue, F.

2026-09-22 health informatics 10.64898/2026.09.21.26363536 medRxiv
Top 0.3%
5.4%
Show abstract

Electronic health records provide a source of real-world information for disease risk prediction, but many machine learning models learn implicit associations that are difficult to inspect or use for clinical reasoning. Causal Bayesian networks (CBNs) offer a form of causal-interpretable machine learning by representing conditional dependencies, putative directional relations, and probabilistic evidence propagation in a directed acyclic graph. However, conventional CBN structure learning becomes unstable and computationally expensive when applied to large-scale EHR data. To address this limitation, we propose UPEBNL, a scalable framework for causal-interpretable cancer prescreening based on parallel CBN learning. UPEBNL integrates adaptive data slicing, quality-aware structure aggregation, and global DAG construction to learn stable and interpretable dependency structures from large-scale observational data. We evaluated UPEBNL in high-dimensional and multi-million-sample simulations and applied it to EHR-based prescreening for esophageal and colorectal cancer. In simulations, UPEBNL improved structural recovery accuracy by nearly 40% and achieved up to a 221.28-fold speedup over conventional CBN learning strategies. For cancer risk prediction, the learned CBNs provided interpretable evidence paths and achieved validation AUCs of 0.8171 for esophageal cancer and 0.784 for colorectal cancer. Calibration and decision curve analyses further supported the reliability and clinical utility of the models. These findings suggest that scalable CBN learning can support interpretable cancer prescreening from large-scale EHR data.

20
AI-Driven Analysis of Device Duration in Anesthesiology

Sutar, A.; Tabman, B.; Fatima, H.; Norouzzadeh, P.; Martins, Y.; Yazdi, M.; Rahmani, B.

2026-09-28 anesthesia 10.64898/2026.09.25.26364018 medRxiv
Top 0.4%
5.1%
Show abstract

Lines, drains, and airway devices are commonly used in surgical and inpatient care to support treatment and patient monitoring. While these devices are essential, prolonged device retention is associated with complications such as infection, tissue injury, thrombosis, and extended hospital stay. Understanding how long devices remain in place and how duration varies across device types is important for improving patient safety and guiding device management practices. This study analyzes device duration patterns using Patient Lines, Drains, and Airways (LDA) data containing information on device type, placement time, and removal time across multiple patient encounters. Exploratory data analysis and machine learning approaches are used to examine duration distributions, distinguish short and prolonged device retention, and identify structural patterns across device categories. The findings indicate notable variation in device duration patterns across categories, with identifiable differences in device retention behavior across device groups. This study highlights the value of observational electronic health record data for characterizing device exposure patterns and provides a foundation for future data-driven device management and risk assessment.